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Abstract 

This paper reports on the development of a tool that generates randomised, 
non-multiple choice assessment within the BlackBoard Learning 
Management System interface. An accepted weakness of multiple-choice 
assessment is that it cannot elicit learning outcomes from upper levels of 
Biggs ’ SOLO taxonomy. However, written assessment items require extensive 
resources for marking, and are susceptible to copying as well as marking 
inconsistencies for large classes. This project developed an assessment tool 
which is valid, reliable and sustainable and that addresses the issues 
identified above. The tool provides each student with an assignment 
assessing the same learning outcomes, but containing different questions, 
with responses in the form of words or numbers. Practice questions are 
available, enabling students to obtain feedback on their approach before 
submitting their assignment. Thus, the tool incorporates automatic marking 
(essential for large classes), randomised tasks to each student (reducing 
copying), the capacity to give credit for working feedback on the application 
of theory), and the capacity to target higher order learning outcomes by 
requiring students to derive their answers rather than choosing them. Results 
and feedback from students are presented, along with technical 
implementation details. 
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Background and Motivation 

In most Australian universities, first year chemistry is assessed largely by multiple choice (MC) 
examinations because of the enormous class sizes (over 2000 students at some universities; around 
850 at the author’s institution). These types of examinations have well-known weaknesses in both 
motivating and assessing quality learning (Nicol, 2007). Flowever, they have the advantage for 
lecturers of being quickly and easily marked for large groups of students, and for students of being 
perceived as objective. If best practices in designing MC tests are followed, they can promote and 
test higher-order learning (Parmenter, 2009); however, it is generally accepted that students do not 
approach learning in a deep way when they know that assessment is MC (Scouller, 1998). 
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First year chemistry involves learning a large number of new concepts (Nurrenbern & Pickering, 
1987), and the students require extensive practice to be able to apply these concepts to different 
fact situations (Pendley, Bretz, & Novak, 1994). In the first year chemistry units at this institution, 
low stakes, progressive assignments are used to give the students the opportunity to practice the 
concepts and to motivate their study. In three of the four first year chemistry units at this 
institution, these assignments are made available on the Learning Management System (LMS) to 
be downloaded by students, submitted several weeks later as hand written documents and then 
hand marked. There are several problems with this: 

• The students often copy answers word for word from each other. This is easily noticed 
when marking, because some strange errors are found on multiple assignments. 

• The marking sessions are very time-consuming (usually 6 people for a 5 hour marking 
session) and relatively expensive due to remuneration rates for sessional staff, especially 
given that the sessions occur multiple times each semester, every semester. 

• Once the assignments have been marked, the marks must then be input into a spreadsheet, 
which is not only time-consuming, it is also subject to human error (particularly when 
several students have similar names). 

• Although students are awarded part marks for showing their working, it is unfeasible to 
ensure consistency in the way part marks are awarded over such a long session with a 
large number of markers. 

• In 2010 and 2011, lecturers in two of the first year units at this institution found several of 
their exact assignment questions asked on websites where students request answers 
(known as Answer Sites). The unique wording of the questions was such that these could 
only have originated from students from this institution (and in one case the student used 
their real name, so this could be verified) (Associate Professor P. Fredericks, personal 
communication, March 20, 2011; Yahoo!7 Answers, 2011). 

Thus, the use of a LMS to provide a computer-marked assignment that offered different sets of 
questions to each student was appealing. This would address the issues of copying (to some 
extent), time and financial costs of marking sessions, and consistency in awarding partial marks in 
hand marking. It does not prevent students from using Answer Sites to attempt to obtain the 
answers to their particular questions (although the quality of answers provided on those sites is 
known to be low (Adamic, Zhang, Bakshy, & Ackerman, 2008). This last problem can be at least 
partially addressed by reducing the time available for completing the assignment. 

Problems with Multiple Choice Assessment for Chemistry 

In university chemistry, students learn a new way of expressing knowledge, through the use of 
chemical symbols and equations. Therefore it is extremely important that they develop and 
practice the fundamental skills of writing and balancing chemical equations. MC questions that 
require only the choice of the correct answer from a list, rather than writing the answer, are 
unlikely to lead to the development of these skills. 

In many cases, the same question can be asked either as MC or as short answer. When MC 
questions are used, basic skills of logical deduction are required, as well as some chemistry 
knowledge. When the correct responses are chemical formulae, equations or numerical answers, 
more and higher order chemistry learning outcomes (LOs) are required within the SOLO 
taxonomy (Biggs & Collis, 1982; Biggs & Tang, 2007). In particular, a correct answer to a MC 
question may indicate a unistructural approach, but such an approach is unlikely to lead to the 
correct response for a short answer question, in the absence of cues or clues in answer choices 
(Schuwirth, van der Vleuten, & Donkers, 1996). To determine the correct response to a short 
answer question, students require at least a multistructural or relational approach, depending on the 
question. The SOLO taxonomy has been used to characterise student learning in organic chemistry 
by a detailed analysis of written answers (Flodges & Flarvey, 2003). Within the tool described 
here, the approach taken by the student can only be deduced from the answer provided, so such a 
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detailed analysis is not possible. However, requiring students to write their answer rather than 
choose it will take them beyond a unistructural approach. 

An example of the type of question, which requires higher order learning if requiring a written 
answer rather than MC, is: 

The products for the reaction of lead nitrate with potassium sulfate in aqueous 
solution are: 

(a) PbS0 4 (aq) + 2 KN0 3 (aq) 

(b) PbS0 4 (s) + 2 KN0 3 (s) 

(c) PbS0 4 (s) + 2KN0 3 (aqt) 

(d) Pb 2+ (aq) + 2 K \aq) + 2 N0 3 “(aq) + SO 4 2 ~(aq) 

(e) none of the above 

It can be seen that the MC options prompt the student with the clues of correct chemical formulae 
and therefore allow a lower order level of learning to enable the correct choice; knowledge of the 
solubility of the products is sufficient, which is unistructural. In contrast, if this question is asked 
as a written question without the five answer choices, the student must understand the theory to a 
deeper level. In the latter case, the student must know (or look up) the correct chemical symbols 
and charges for the ions, which are both essential LOs for first year chemistry, and relate these to 
the solubility of the compounds, in a multistructural approach. Many variations on this particular 
question can be asked by altering the two metals and the two anions. 

Other examples of questions that require higher level learning when asked as short answer 
compared to MC are any requiring numerical calculations. When choices are offered, the students 
are given at least one answer with the correct value and the correct order of magnitude. It is 
therefore easier for students to work out which choices are incorrect than to obtain the correct 
numerical value with no prompting. This is apparent from reading the answers given by students in 
both hand written and online assignments - some bizarre answers are reached, which would never 
have been given as an option in a MC setting because they are so unrealistic. Thus, the feedback to 
lecturers on student misconceptions through an analysis of incorrect written responses is also 
lacking in MC assessment. 

A final problem with MC questions is that they can make it difficult for outstanding students to 
excel. This is because it is very easy to accidentally enter the wrong choice, even when the correct 
answer has been obtained. 

Aim of this Project 

In light of the background above, it was desirable to develop assignments for a first year chemistry 
unit that would be computer-marked but not MC. This institution uses BlackBoard as a LMS, and 
this package is capable of automatically marking answers in numerical or text formats. It is also 
capable of assigning students different sets of questions randomly from pools. Thus, the aim of 
this project was to use the full capability of the LMS to develop non-multiple choice, randomised 
assignments for a new unit that was run for the first time in 2008. Feedback from students was 
used to improve the assignments in subsequent semesters and the details are presented below. 
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It is worth noting that many modem text books have online components, in which the publishers 
have set up an enormous variety of sophisticated non-MC questions that can be assigned to 
students for formative and summative purposes. Two popular examples are from Wiley (2011) 
and Pearson (2011). However, for this project it was thought that it is better to remain within the 
internal LMS. This allows questions to be tailored to the exact content covered at this institution, 
and enables the questions to be reused regardless of changes to text books. It also allows the 
questions to be independent of the students having purchased a particular text book or signing up 
for an online resource. The policy of the author’s institution is that while externally hosted sites are 
not recommended for assessment, they are permitted. 

The assignments form only a small portion (2 x 5%) of the total assessment of the unit involved, 
which is also assessed by practical laboratory reports (10 * 5%), a progress examination (10%) 
and a final examination (30%). The goal of the assignments is to encourage students to engage 
with the theoretical material to support their learning in the laboratory and to obtain feedback on 
their progress throughout the semester. Within the SOLO taxonomy (Biggs & Collis, 1982), a 
relational or extended abstract approach is then required of students to relate the theory in the 
assignments to their laboratory work. 

Design and Implementation 

Design and Initial Implementation 

The design of the online assignments started with the written assignments used in other first year 
chemistry units as a model. In those units, two or three assignments are given, each containing 10 
questions, and each worth 5% of the final mark. The students have three or four weeks to complete 
each assignment from when they are made available on the LMS until the date that they must be 
submitted, as paper documents. 

In this project, the LMS was intended to mark the assignments, so the answers could only be a 
number, a single word or a chemical formula. These answer options allow the students to enter 
exactly what has been set as the correct answer. The content of first year chemistry lends itself 
well to questions with such answers, because students learn to perform a large variety of simple 
calculations and become familiar with many simple chemical formulae. For numerical answers, 
the LMS allows a range of numerical answers to be marked correct. For example, if the correct 
answer is 10.2, the question can be written to accept anything between 10.1 and 10.3, or between 
10.15 and 10.25, as specified in the question. Thus, if a student answers 10.19, they will be marked 
correct. In some cases, it is more appropriate to write the question such that only the exact answer 
is accepted, depending on the theoretical basis of the question and the sources of uncertainty. For 
text answers, it is possible to specify as many answers as necessary to be correct if there are 
possible ambiguities in how the answer should be typed. For example, if the correct answer is the 
name of the chemical compound 1,2-dimethylbenzene, the common name of this compound, 
ortho-xylene, must also be specified as a correct response unless the students are aware that only 
formal names are acceptable. 

The use of the LMS allowed each question to be written with multiple variations to form a pool for 
each question type. In this project, around 15 variations were written for each question. For 
questions with numerical answers, the question variations were generated reasonably easily using 
Excel. Figure 1 shows a screenshot of a question generator that was designed by the project team. 
On the page that is shown, the question requires determination of the percentage yield of a 
reaction. To obtain the correct answer, the students must first write a balanced chemical equation 
for the reaction. They must then determine the limiting reagent and maximum theoretical yield 
before they can determine the percentage yield obtained in their version of the question. 
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Figure 1: Screenshot of the Excel document used to generate numerical questions; 
the page shown is for Assignment 2, question 5 


For each question type, the Excel page must be manually designed and checked, which requires a 
significant amount of time. For the page shown, for example, columns A, B, D, E, I and M are 
chosen by the programmer. Columns A, E and I are the variables for the different questions in the 
pool. Columns C, F, G, H, J, K, N, O and P are calculated by underlying formulae within the Excel 
spread sheet, depending on the variable values. Columns L and P are the two answers required by 
this particular question in the assignment, where column L must be manually input by the 
programmer by comparing the calculated values in columns G and K. 

For qualitative chemistry questions, the questions must be individually written, which is also time 
consuming. It is particularly difficult to ensure that the difficulty of all questions is comparable for 
non-numerical answers; this may impact on the reliability of the assessment. An illustration of this 
is if the question requires the identification of an organic compound. Small compounds such as 
methanol and ethanol might be considered easier to identify than more complex molecules such as 
2-pentanone. This limits the number of possible different questions in the pool. 

Once the questions have been written, the question pools themselves must be written within the 
LMS. Because the text of each question within a pool is virtually identical, it is relatively quick to 
make 15 copies of the question and then edit each question individually to input the variable 
values and the correct corresponding answer(s). An example of the resulting pools is given in 
Figure 2. 
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Calculated Numeric: The standard cell potential tor the... 

Question The standard cell potential for the voltaic cell based on the reaction below is_V. (Provide your answer to 

2 decimal places.) E 0 values are from the prescribed text book (appendix F page A-7 & A-8). 

Cu( s) + 2HOCI( aq) + 2H *( aq) — Cu 2 *( aq ) + Cl 2 ( g) + 2H 2 0( /) 

Answer 1.29 

Answer range +/- 0.01 


Calculated Numeric: The standard cell potential for the... 

Question The standard cell potential for the voltaic cell based on the reaction below is_V. (Provide your answer to 

2 decimal places.) E 0 values are from the prescribed text book (appendix F page A-7 & A-8) 


Answer 

Answer range +/- 


0.31 
0 01 


Calculated Numeric: The standard cell potential for the... 

Question The standard cell potential for the voltaic cell based on the reaction below is_V. (Provide your answer to 

2 decimal places.) E 0 values are from the prescribed text book (appendix F page A-7 & A-8). 


3F 2(g) + 2Mn0 2 (s) + 4H 2° (I)' 
Answer 

Answer range +/- 


-6F- (aq) + 2Mn0 4 - (aq) + 8H* (aq) 


1.36 
0 01 


Calculated Numeric: The standard cell potential for the... 


Question The standard cell potential for the voltaic cell based on the reaction below is_ 

2 decimal places ) E° values are from the prescribed text book (appendix F page A-7 & A-8) 


2HOCI + 2H * + 2Fe 2 * - 

(aq) (aq) (aq) 

Answer 

Answer range +/- 


■ C ' 2 (g ) + 2 H 2 0( 1) *2Fe 3 * (aq) 


V. (Provide your answer to 


0 86 
0 01 


Calculated Numeric: The standard cell potential for the... 

Question The standard cell potential for the voltaic cell based on the reaction below is_V. (Provide your answer to 

2 decimal places.) E° values are from the prescribed text book (appendix F page A-7 & A-8) 


Figure 2: Screenshot of question pool for Assignment 2, question 2 


It is worth noting that the LMS does have the ability to itself randomly generate questions, given a 
range of values for a variable. This facility has been used in other disciplines at the author’s 
institution (Dr G. Isoardi, personal communication, April 4, 2011). However, for the specific 
questions used in this project, the function was not used because we wanted to be sure of the exact 
numerical values that the students would be asked. 

The content covered, answer type and marks available for each question in the two assignments 
are given in Table 1. 
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Table 1: Content, answer type and marks for each question in each Assignment 


Assignment 1 

question content 

answer type 

marks 


1 

accuracy 

yes/no 

5 


2 

precision 

yes/no 

5 


3 

error analysis 

numerical 

10 


4 

significant figures 

numerical 

10 


5 

solubility (qualitative) 

increase/decrease/unchanged 

5 


6 

solubility (quantitative) 

numerical 

10 


7 

thermochemistry 

numerical 

10 


8 

kinetics 

numerical 

15 


9 

integrated rate laws 

numerical 

20 


10 

pH 

numerical 

10 


Assignment 2 




question 

content 

answer type 


marks 

1 

oxidation numbers 

whole number 


10 

2 

electrochemical cell potential numerical 


10 

3 

Nernst equation 

numerical 


10 

4 

chromatography 

faster/slower 


5 

5 

percentage yield 

numerical 


10 

6 

limiting reagent 

chemical name and numerical 

10 

7 

empirical formula 

chemical formula 


10 

8 

NMR spectroscopy 

whole number 


10 

9 

mass spectrometry 

chemical formula 


10 

10 

identification of an unknown chemical name 


15 


Within the LMS, each student then received a randomly generated assignment consisting of one 
question out of each pool. An example of the appearance of the assignment as seen by the students 
is shown in F igure 3. 
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Do these results show good accuracy? yes or no 

I I 

^ Question 2 5 points Save Answer 

When measuring the temperature of a room from three digital thermometers you obtain the values of 22.4. 28.3 and 27.6. 
The true temperature is 25.5- 

Do the digital thermometers show good precision? yes or no 

I I 

^ Question 3 10 points Save Answer 

In the titration of 25.00 ± 0 04 ml. of 1 05 ± 0.005 mol L' 1 hydrochloric acid, 19.45 ml_ of sodium hydroxide was required for 
neutralisation. The absolute error in the NaOH molarity is_mol/L. 

I I 

Question 4 10 points Save Answer 

The result for the following molecular mass calculation 4 x 15.9994 + 32.066 + 2 x 1.0079 to the correct number of significant 
figures is_ 


0 Question 5 5 points Save Answer 

Would the addition of KOH to a solution containing Cu(0H) 2 increase, decrease or not affect the molar solubility of Cu(0H) 2 ? 

I 

Figure 3: Screenshot of part of Assignment 1 in the student view 


The students had several weeks from when the assignments were available on the LMS to 

complete and submit their assignment. They were encouraged to print out their own assignment 

and to work in groups if desired to determine how to solve the questions. 

The instructions provided with each assignment are as follows: 

1. The Assignment is on the BlackBoard site under "Assessment", "Assignment Number". Once you click 
"OK" to begin the Assignment for the first time, you will get your Assignment. Every student will have 
a different Assignment. 

2. You can open the Assignment and look at the questions as often as you want. It is strongly suggested 
that you print it out to work on it. You may save your answers as you go and return to complete the 
Assignment later. You can also change saved answers. 

3. All answers are one of the following: 

a number, 

a chemical formula, 

a single word ("yes", "no", "increase", "decrease") 

You do not need to provide units with any answer. For chemical formulae, you do not need to write 

subscripts or superscripts, but you must write brackets where required. Do not leave any spaces in your 
chemical formulae or the software will not recognise your correct answer, e.g. iron (II) nitrate is to be 
written Fe(N03)2 

4. For numerical answers, be careful with the number of significant figures. Providing too many 
significant figures is wrong, just as providing not enough significant figures is wrong. Exponential 
notation should be entered as follows: 

3.4 xlO ' 6 is written 3.4E-6 into the BlackBoard assignment. 
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5. Not all of the questions are worth the same number of marks; more time-consuming questions (e.g. you 
may need to plot data or solve an equation) are worth more marks. The Assignment is marked out of 
100, and it is worth 5% of your final mark for SCB131. 

6. When you have finished and have answered all the questions, click on "submit". YOU CAN ONLY 
DO THIS ONCE. Once you submit your answers, you will receive your marks and a list of which 
questions you got right and wrong. 

7. If you have any technical problems whilst using BlackBoard, ring 3138 4000 or email ithelpdesk@ 
qut.edu.au immediately and report it. 

8. Assignments may not be submitted after the due date. Assignments close at 5 pm on the due date. It is 
your responsibility to ensure that you attempt the Assignment before the due date; if you miss the 
Assignment for any reason you will not be given another opportunity to submit it. 

9. It is suggested that you attempt the Assignment from a QUT computer. Failure to submit the 
Assignment due to an inability to access the Assignment from a non-University computer is NOT an 
acceptable excuse. 

10. Keep your receipt upon completion of the Assignment in case there is any problem with your mark. 

11. It is strongly recommended that you not leave it until after 4.00pm on the due date to attempt the 
Assignment in case you experience any technical difficulties. You will still be expected to attempt the 
Assignment before the original deadline. 


Student Feedback on Initial Implementation 

The tool was used for the first time during the first iteration of this unit in semester 2, 2008. In the 
first implementation, there were no practice questions and the student feedback strongly indicated 
that the students found this difficult. Of the 61 written responses within the end of semester 
Student Experience Survey to the question “What aspects of this unit are most in need of 
improvement?” 10 addressed the online assignments. Those (unedited) comments from the 
students relating to the assignments were: 

• The online quiz was pathetic i got all the answers right hut I didn't get the marks due to 
rounding error which wasn't taught into get depth. 

• Online assessment - no credit given for staged working out and small margin for error ie 
right/wrong answers 

• Assignments- often extremely difficult and online submission seemed unfair as marks could be 
lost very easily. 

• I think that assignments need to be mark by persons. Submitting on line is like a 50/50 chance. 
As well as we don't have mark for our working out etc. 

• The online assignments were a complete waste of time and did not accurately assess student's 
knowledge or abilities. It seems amazing to me that I can spend days working out a particular 
calculation, and lose the ENTIRE 20 marks (out of 100) because I put the Ph at 8.88 instead 
of 8.85??? 

• The online assignment format was horrible, many many student lost a large amount of marks 
for answers the computer did not register because of rounding at different points (the amount 
of error she gave was . 000001 for a question with only 3 significant figures.) 

• the assignments were not particularly well thought out or very helpful in the end. It was 
difficult to determine where I had gone wrong in the questions I answered incorrectly & since 
part marks were not available for correct working etc the was exaggerated. 

• The online assignments need some attention. I have done online assignments before and it was 
a much better system. I realise that chemistry requires a high level of accuracy but it was a 
little frustrating to get marked completely wrong for rounding errors. 

• The online assignment definitely needs to be re-evaluated as I was not happy with getting a 
question wrong when my answer was only . 005 off the correct value and I didn't receive the 
notice regarding carrying sig. fig's all the way through til later. 

• Assignment on blackboard is in dire need of review!!! 

Only one student response relating to the assignments was received among the 57 responses to the 
question “What were the best aspects of this unit?” as follows: 

• Online Assessment was a good inclusion to the unit 
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In semester 1, 2009, a different first year chemistry unit at the same institution that normally uses 
paper assignments trialled a similar online assignment for one of the three assignments in that unit. 
Students were surveyed regarding which assignment format they preferred; 61 out of 67 
respondents preferred the paper assignments. The following (unedited) comments regarding the 
online assignment were received from those who preferred the online format: 

• This assignment was much better became feedback was given straight away in preparation 
for the final exam 

• It’s good as I didn't have to worry about units or making sure my working out was legible. It 
was also easier as I didn't have to hand it in manually 

• Easy to complete - different questions for every student which was good. 

• Easy to summit, no mass on the paper, save the paper for ecosystem. 

• It's good to have an assignment that's different to everyone else's. It's also easier to submit! 

• It lets the students complete the assignment to their full extent rather than cheating off the 
paper assignment. 

• They are easy to work on, if I need to write questions out I still can, it's easy to save and try 
again later. It's great getting a mark back straight away! 

• I do like online assignments, only because you get your grade instantly. 

Among the students in that unit who preferred the paper format, the overwhelming reason was that 
part marks were awarded for working, so even if their final answer was wrong they would receive 
some marks for that question. Students understandably were very upset at losing all the marks for 
a question simply because they put too many digits in their answer (for example). It is clear from 
these comments that the instant feedback of the online assignments is seen as a great benefit 
compared with hand marked assignments, for which there is usually a delay of around two weeks 
between submission and receiving their marks. It is worth noting that for these large classes, the 
hand marked assignments do not include detailed individual feedback, although detailed worked 
solutions are provided. 

Further Implementation of the Assessment Tool 

In response to the negative feedback from students regarding being given credit for their working, 
for the second iteration of this unit in semester 2, 2009, an extra set of each pool of questions was 
written, which required a moderate input of time. Extra numerical questions can easily be 
generated, but questions involving chemical compounds are more difficult, because it is important 
that all the compounds be real substances and that the questions be of similar difficulty. There is 
no overlap between the practice pools and the pools of questions for the summative assignments. 
The students were then given access to formative assignments based on the new pools. Students 
were able to attempt the practice assignments as often as they wanted. 

Provision of the formative assignments vastly improved the student feedback on the summative 
assignments, as seen from the following end of semester Learning Experience Survey feedback, 
where four out of 57 responses to “What were the best aspects of this unit?” addressed the online 
assignments: 

• I really liked the practice assignments. They were very helpful in helping me to determine the 
correct way to give the answer. 

• The prac assignment was awesome, great to be able to check your reasoning before 
submitting the assignment was a huge bonus. 

• Assignment practices were good 

• The assignments were the best aspect of the unit. 

However, five out of 59 responses to “What aspects of this unit are most in need of 
improvement?” addressed the online assignments, again with the complaint that part marks are not 
awarded for working, and no marks are awarded if there is a small error in a numerical answer. 

The range of acceptable answers can be further widened to address this, as well as reminding 
students that their working on the formative assignments results in better outcomes in the 
summative assignments. 
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The practice assignments are very useful to the academics involved, because it is possible to look 
up individual student attempts on the formative assignment. All details can be accessed, including 
what answers the student gave, how many attempts they made and when the attempts were made. 
Thus, if a student queries an aspect of the summative assignment, it is possible to check whether 
the student took advantage of the formative assignment before submitting their summative 
assignment. 

It is possible to directly compare the outcomes of students in the two assignments over the two 
years, without (2008) and then with (2009) the formative assignments; Table 2 contains this data. 
The questions were not changed and the student cohort was similar each year. 

Table 2: Summative assignment results without and with formative assignments 


Year 

Assignment number 

Number of students 

Mean ( SD) 

2008 

1 

158 

49.59 (22.55) 

2008 

2 

152 

65.07 (22.37) 

2009 

1 

245 

58.82 (26.27) 

2009 

2 

241 

65.33 (25.57) 


The marks for Assignment 1 over the two years were significantly different; f(401) = 3.64. p < 
.001, with the students that had access to the formative practice assignments in 2009 receiving 
higher marks. In contrast, the marks for Assignment 2 were identical over the two years. This is 
most likely due to the types of responses in the two assignments (Table 1). Assignment 1 requires 
mostly calculated numerical answers, for which the students can check their procedure using the 
formative Assignment. By comparison, most of the answers in Assignment 2 are chemical names, 
formulae and whole numbers. The underlying theory to determine the correct answers for these 
questions is more conceptual, and access to practice questions apparently makes less difference. 

The concept of sustainable assessment has been developed by Boud (2000, p. 152). He states that 
for assessment to be sustainable, students should be able to “identify whether they have met 
whatever standards are appropriate for the task in hand and seek forms of feedback from their 
environment (from peers, other practitioners, from written and other sources) to enable them to 
undertake related learning more effectively.” 

By attempting this formative assessment and checking their results, students are able to determine 
whether they are meeting the standard for the assessment task before submission of the summative 
assignments. Thus, provision of the practice assignments was very important to make the 
assignment assessment sustainable, and enables the students to learn more effectively. 

Weaknesses of the LMS for Online Assessment of Chemistry 

There are several technical problems with the use of the LMS to administer the questions, which 
diminish the value of this form of assessment compared with written and hand marked assessment. 
These relate to the input of numerical values and units and are explained in detail here. 

In first year science, one concept that students are learning is the extremely important concept of 
measurement uncertainty, which encompasses error, accuracy and reporting significant figures. 
Thus, they learn that it is incorrect to specify more decimal places on an answer than can be 
experimentally measured. For example, in a titration, the concentration determined could never be 
correctly written as 0.192549 molL' 1 , because the equipment used simply does not allow that level 
of accuracy. Therefore, values must be given to the correct number of significant figures, such as 
0.19 molL" 1 . However, if the LMS accepts a range of answers (for example, 0.18 - 0.20 molL' 1 ), 
the response 0.192549 molL' 1 would be marked correct by the LMS because it lies numerically 
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within that range. This response would in fact lose marks in a hand marked assignment for giving 
excessive significant figures. To obviate this problem, it is possible to specify in the LMS no range 
of values, so that the students must give the exact correct numerical answer in order to be marked 
correct. However, in that case they may use a slightly different rounding technique or value for a 
physical constant at some stage of their calculation, and thereby legitimately obtain a slightly 
different final value. Then they are marked incorrect by the LMS, although for a hand written 
assignment, they would be given most or all of the marks. 

A second problem with numerical answers within the LMS is that for the computer to recognise 
the numerical response as a value within the range specified as correct, the students cannot give 
units. Learning to give the correct units for a measurement (such as mL, molL" 1 , bar etc) is a very 
important part of first year science, so not requiring units to be given in the answers is a significant 
weakness of the LMS with respect to the LOs. This can be avoided by having the students input 
their answer as text rather than having it be treated as a numerical value, because the text field can 
be anything specified. The problem then arises that the numerical answer and units must be given 
exactly as specified or the answer will be marked incorrect. For example, in the above titration, the 
answer could require the student to input 0.19 molL-1 to be marked correct. If the student types 
more or less spaces, or gives a slightly different value of 0.18 or 0.191 molL-1, they will be 
marked incorrect, although they have understood and correctly used all aspects of the theory. 

In our current implementation, the units for numerical responses (such as mL, V, molL' 1 ) are 
provided to the student as part of the question (as seen in Figure 3, question 3) and they are only 
required to input their numerical response, so that the value can be recognised by the LMS as lying 
within the correct range. Thus, students giving an excessive number of decimal places are not 
being penalised, and students are not required to know the units for their answers. As can be seen 
in the student feedback, at least some of the students recognise that this is an easier option than 
requiring them to input the units themselves, and in fact providing the units may act as a cue to 
prompt their response. This issue will be addressed in future implementation of the tool, so that 
students are also tested on their understanding of units. This can be done by using multipart 
questions, so that the units must be entered by students separately from their numerical responses. 
A final, minor problem with any online assessment is that students occasionally make a genuine 
typographical error, for example inputting units when these are provided. It is relatively easy to 
correct individual marks if the students let their instructor know that this has occurred. With large 
student numbers, the time required to make the minor corrections increases, and there is always the 
risk that a student does not inform the instructor so they do not receive their due mark. The 
relatively small weighting of these two assignments (5% each) makes this possibility unimportant. 

Conclusions 

The tool developed in this project requires extensive resources to set up: around 6 hours per 
question for writing, inputting to the LMS and checking, a total of at least 120 person hours. 
However, once in place, it requires minimal effort each semester. In comparison, hand marking 
and data entry takes around 65 person hours per semester, so within two years this tool should be a 
worthwhile investment of time. Students appreciate that their classmates are not able to cheat and 
that they receive their results instantly upon submitting their assignments. The advantages of 
instant marking, automated mark entry, provision of unlimited practice questions and avoiding 
cheating due to randomisation outweigh the weaknesses described above. The questions asked in 
the assignments are aligned with the teaching activities and intended LOs of the unit, making them 
a valid form of assessment according to Biggs (2003). They are reliable, because the LMS is 
consistent in its marking. Sustainability is achieved through the use of the practice assignments, so 
that students are able to self-monitor their own progress (Boud, 2000). Thus, we have developed 
and used a form of assessment that is valid, reliable, sustainable, and takes advantage of the power 
of the modern LMS. It fulfills the requirements stated in the Australian Learning and Teaching 
Council document Assessment 2020 (Boud and Associates, 2010). 
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